With the advance of computer and network technologies, digital images and videos have become commonplace in people's personal and professional life. Photographs taken with digital cameras and exchanged through email or cellular phones, video conferencing, digital TV broadcasting, pay-per-view Internet services, to name just a few, are getting more popular everyday. With their popularity, comes the attention to technologies for improving image and video's quality and reducing their size for easy transmission and exchange. Denoising is such a technology, which is a process taken to remove noise from digital images and videos and to enhance their compressibility.
Noise usually exists in uncompressed images and videos, which are unwanted or undesirable elements or artifacts. Noise can come from a variety of sources, for example, shot noise which originates in the electronic hardware, thermal noise, channel noise, etc. The existence of noise not only degrades the visual quality of video, it also decreases the performance of video coding and reduces the medium files' compressibility. It increases the entropy of the video such that more bits are needed to code the images and videos and also decreases the accuracy of motion estimation that further increases the bit rate. To increase the coding efficiency and compressibility, video denoising is necessary before encoding the video. The purpose of video denoising is to estimate the true image signals as accurately as possible. Many methods have been developed for image denoising such as low pass filtering, Wiener filtering, Kalman filtering or Spatial Varying filtering, most of which exploit the spatio-redundancy of the image to suppress noise. However, these filters are not optimized, especially for video denoising as the temporal redundancy of the video is not exploited. Some spatio-temporal filters are proposed for denoising where motion compensation is usually applied in order to exploit the temporal redundancy of moving objects. However, the computational complexity of these methods is too great for real time processing. Spatial Varying Filter (SVF) was proposed, which has relatively low computational demand and yet good edge preserving capability. SVF computes filter weights by the difference between current pixel and unfiltered neighboring pixels. Because SVF is a Finite Impulse Response (FIR) filter, the filter size is limited due to computational complexity. This constraint limits the noise suppression capability of the SVF technique as the order of the filter cannot be too large.
Some references are briefly commented in the following for purpose of providing background information.
U.S. Pat. No. 6,731,821 discloses an adaptive low pass filter to remove noise in digital video signals. The amount of low pass filtering depends on a local gradient measurement. The image is first filtered by a general low pass filter and then the filtered image is subtracted from the original image to give the residue image. The residue image is weighted according to local gradient measurement and added back to the filtered image to give the final image. However, the weighing of the neighboring pixels used in the low pass filter is not adaptive.
U.S. Pat. No. 6,665,448 discloses a filter which performs selective sharpening and smoothing of image data. The sharpening filter is derived from the selective smoothing filter. Unsharp masking is performed to enhance the edges. The purpose of the filter, however, is to enhance the visual quality of the image with enhanced edges which, in general, increase the bit rate.
U.S. Patent Application No. 20050036704 teaches a method of splitting an image into the foreground region and the background region and applying different anisotropic diffusion filters on each region. But, in general, a video scene is more complex than just two regions.
U.S. Patent Application No. 20050036558 teaches a method where pixel values between successive frames are filtered if the difference in the pixel values between successive frames is within threshold range, which is determined adaptively frame by frame. The use of fixed threshold within a frame nonetheless limits the performance of the filter.
U.S. Patent Application No. 20020172431 discloses a method for improving the appearance of the image while at the same time enhancing the compressibility of the image. It improves compressibility by selectively filtering the smooth regions and enhances the edges by applying an edge enhancement filter selected based on an edge analysis. However, the method only uses a limited set of filters.
U.S. Pat. No. 6,633,683 discloses a method to improve a well-known “least mean square method” by locally segmenting the pixels to the homogeneous region and edge region and estimating the local mean and variance using the data only from the homogeneous region. This method, however, requires the knowledge of noise variance to give good performance and it is for handling Gaussian noise.